A measure of statistical complexity based on predictive information

We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s predictive information, and the multi-information. We derive some of the properties of the binding information, particularly in relation to the multi-information, and show that, for finite sets of binary random variables, the processes which maximises binding information are the 'parity' processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.Comment: 4 pages, 3 figure

Towards music perception by redundancy reduction and unsupervised learning in probabilistic models

PhDThe study of music perception lies at the intersection of several disciplines: perceptual psychology and cognitive science, musicology, psychoacoustics, and acoustical signal processing amongst others. Developments in perceptual theory over the last fifty years have emphasised an approach based on Shannon’s information theory and its basis in probabilistic systems, and in particular, the idea that perceptual systems in animals develop through a process of unsupervised learning in response to natural sensory stimulation, whereby the emerging computational structures are well adapted to the statistical structure of natural scenes. In turn, these ideas are being applied to problems in music perception. This thesis is an investigation of the principle of redundancy reduction through unsupervised learning, as applied to representations of sound and music. In the first part, previous work is reviewed, drawing on literature from some of the fields mentioned above, and an argument presented in support of the idea that perception in general and music perception in particular can indeed be accommodated within a framework of unsupervised learning in probabilistic models. In the second part, two related methods are applied to two different low-level representations. Firstly, linear redundancy reduction (Independent Component Analysis) is applied to acoustic waveforms of speech and music. Secondly, the related method of sparse coding is applied to a spectral representation of polyphonic music, which proves to be enough both to recognise that the individual notes are the important structural elements, and to recover a rough transcription of the music. Finally, the concepts of distance and similarity are considered, drawing in ideas about noise, phase invariance, and topological maps. Some ecologically and information theoretically motivated distance measures are suggested, and put in to practice in a novel method, using multidimensional scaling (MDS), for visualising geometrically the dependency structure in a distributed representation.Engineering and Physical Science Research Counci

Analysing symbolic music with probabilistic grammars

Recent developments in computational linguistics offer ways to approach the analysis of musical structure by inducing probabilistic models (in the form of grammars) over a corpus of music. These can produce idiomatic sentences from a probabilistic model of the musical language and thus offer explanations of the musical structures they model. This chapter surveys historical and current work in musical analysis using grammars, based on computational linguistic approaches. We outline the theory of probabilistic grammars and illustrate their implementation in Prolog using PRISM. Our experiments on learning the probabilities for simple grammars from pitch sequences in two kinds of symbolic musical corpora are summarized. The results support our claim that probabilistic grammars are a promising framework for computational music analysis, but also indicate that further work is required to establish their superiority over Markov models

A Comparative Earthquakes Risk Assessment Approach Applied to the United Arab Emirates

This paper presents the preliminary results of a long research project on the assessment and mitigation of seismic risk in major cities in the UAE. UAE’s earthquake activity has long been recognized as one of the lowest in the world. All cities have experienced moderate earthquakes in the past, and will again do in the future. Recent earthquakes in Iran (e.g., Bam in 2003, --- in 2005) have killed thousands of people. Because of the different design and construction practices, different population density concentrations and economic activities in the UAE, different damages and losses are likely to be experienced. The impact of an earthquake is not limited to direct losses, such as the loss of life, loss of structures and business interruptions. Earthquakes also cause indirect losses by producing supply shortages and demand reductions in various economic sectors. In a country such as the UAE, which is undergoing an unprecedented constructionbased development with high-rise buildings being the main feature, a large earthquake in a major city can actually cause a considerable economic loss. In this study, a framework for assessing and comparing the risk associated with the adverse consequences of earthquakes in the UAE is presented. The framework is based on a simple risk-characterization model that is used to assess the health risks associated with toxic chemicals. The model: Risk = D × RF × Pop × ER, adopted to fit our purpose of estimating the risk associated with the consequences of earthquakes, the various parameters in the above mentioned model are translated as follows: Dose (D)= seismic “force” at a specific location or weighted for an area; Response Factor (RF) = degree of damage or losses per unit “force”; Population (Pop) = a factor representing exposed population. Equivalent populations may also include exposed environment or exposed infrastructure. Emergency Response (ER) = effectiveness of available emergency response programs to reduce risk immediately as the adverse effects take place. It should be noted that emergency response in this case is different than deliberate risk management. First, the earthquake hazard and risk in the UAE, including the estimation of the amplitudes of the ground motion parameters, is stochastically assessed. Then the comparative risk framework to assess the relative impacts on people and buildings in the seven emirates and the major cities of the UAE is applied. The result is a ranking system for risk that is being integrated within a geographic information system (GIS). The database is intended for detailed development to maximize benefits to the various stake holders in the community

Finite-Difference Time-Domain Simulations of Light Scattering from Retinal Photoreceptors

Recently, a novel optical imaging technique was successfully used in measuring the functional response of living retinal tissues. The technique, functional ultra high resolution optical coherence tomography, measures localized differential changes in the retina reflectivity over time resulting from external white light stimulation. This result can be used to develop a non-invasive diagnostic method for the early detection of retinal diseases. However, the physiological causes of the experimentally observed optical signals, most of which originate from the photoreceptors layer, are still not well understood. Due to the complexity of the photoreceptors, using purely experimental methods to isolate the changes in light reflectivity corresponding to individual physiological processes is not feasible. Therefore, we have employed the finite-difference time-domain method to model the changes in light scattering patterns of the photoreceptor cells caused by light-induced physiological processes. Processes such as cell swelling, cell elongation and hyperpolarization of doublelipid membrane structures were simulated by changing the size parameters and optical properties of the cells components. Simulation results show that the hyperpolarization of double-lipid membranous structures and cell swelling are the most likely causes for the experimentally observed changes in optical reflectivity. A number of experiments were suggested to verify the conclusions drawn from this numerical work. This numerical work includes an analysis of various errors in FDTD computational models

A measure of statistical complexity based on predictive information with application to finite spin systems

NOTICE: this is the author’s version of a work that was accepted for publication in 'Physical Letters A'. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in PHYSICAL LETTERS A, 376 (4): 275-281, JAN 2012. DOI:10.1016/j.physleta.2011.10.066